Contemporary Issues In Education Research - First Quarter 2015 


Volume 8, Number 1 


An Analysis Of Teacher Effectiveness 
Related To Technology Implementation 
In Texas Secondary Schools 

Andrea Kuyatt, Bishop CISD, USA 
Glenda Holland, Texas A&M University-Commerce, USA 
Don Jones, Texas A&M University-Kingsville, USA 


ABSTRACT 

In the age of accountability, schools are struggling to reach every student to bring them up to the 
high academic standards mandated by the federal and state governments. Teachers must use 
research-based strategies to help decrease the learning gap for all students (Kimmelman, 2006). 
While the use of technology to engage students is a popular tool, how teachers use the technology 
available to them may determine the effectiveness of the lesson for the students. State assessment 
scores measure student success and the effectiveness of the instruction they are receiving. 
Technology may be a means to increase that effectiveness to take more students to a deeper level 
of understanding. 

An analysis of variance was used to evaluate the data from surveys on the ways technology was 
used in the classroom and the level of implementation of use in the classroom. There was a 
statistical difference on test scores for teachers who used technology in the classroom and those 
who did not. The study showed the teachers who implemented technology in their classrooms had 
lower scores than teacher who did not. The ability to implement technology in student lessons and 
the use of content standards with technology may change student engagement in the classroom 
(Harris & Hofer, 2011). This information could be significant in helping administrators 
determine successful instructional strategies for their schools. 
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INTRODUCTION 


AA - "/' he ability of the teacher to use technology in a way that will improve student learning and 

C // participation in educational activities has been the basis for studies conducted in recent years 

(Overbay, Mollette, & Vasu, 2011). The introduction of the No Child Left Behind (NCLB) Act 
brought technology to the forefront with the requirements of Title II, Part D, enhancing education through 
technology and increasing pressure in the accountability rating for all schools (Texas Education Agency, 2012). 
According to Goe (2007), effective teaching may be measured by many factors such as qualifications and 
experience, high expectations, classroom environment and the ability to motivate students. While research is 
conducted to narrow the focus of effective teaching strategies many legislators rely more on evidence such as state 
and federal teaching credentials that are obtained through testing to prove competence than the measurement of the 
ability of the teacher to implement quality instruction (Stumbo & McWalters, 2010). Goe (2007) states, “there is not 
a clear consensus that any of the measured characteristics have a clear impact on student achievement” (p. 32). With 
so many schools under sanctions for student performance and administrators who strive to meet the federal 
requirements to keep their schools as top performing, administrators and school boards turn to student assessment 
scores as a measurement of teacher effectiveness. 
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The current educational system is driven by high stakes accountability. Schools are continually searching 
for effective solutions that will eliminate educational barriers and lead all students to success. Texas Education 
Agency (TEA) is the state monitoring division that helps guide programs in public education (Texas Education 
Agency, 2012). TEA provides schools with monitoring and assessment results for Texas public schools based on 
student performance, student groups, and financial divisions. The student assessment reports posted to the TEA 
website include Adequate Yearly Progress Report (AYP), which is a federal report; the Academic Excellence 
Indicator System report (AEIS), the state version of AYP; and the Performance-Based Monitoring Analysis System 
(PBMAS) report which measures student accessibility and performance of subgroups (Texas Education Agency, 
2012). Each of these reports indicates overall student performance data yet fall short of assisting teachers and 
schools in understanding the individual impact of teacher practices that may influence the testing outcomes. 

Administrators look to teacher quality to achieve these goals and search for methods on how to turn 
ineffective teachers into successful partners. Historically, teachers who have many years of teaching experience 
have been the cornerstones of school leadership. These teachers have mastered classroom management, curriculum 
sequencing, and instructional practices. However, with new technology and youth who are proficient enough to 
master multi-tasking and technological media, teachers who are more adept at the new technology age are fast 
becoming personnel who are in high demand with school districts. Watching a teenager play video games or text is 
an example of an engaged student. Teachers who take advantage of the student preferred mode of instructional 
delivery will engage and reach the students of today (Rosen, 2011). 

Current research identifies many aspects of teacher effectiveness. Studies on instructional practices, 
curriculum alignment and school culture have been identified as contributing to teacher effectiveness (The Gates 
Foundation, 2012). Unfortunately, the structure of our teacher effectiveness system has not examined the degree in 
which teachers implement technology and to the capacity in which teachers take advantage of the instructional and 
learning tools they have at their disposal. 

As schools search for ways to meet the demands of No Child Left Behind’s academic standards, they turn 
to teacher effectiveness criteria to determine how to help meet the needs of students. Researchers have differing 
opinions on specific teacher qualities and their effects on student performance (The Gates Foundation, 2012). Few 
studies have been conducted to determine if teachers are using technology as a learning tool as opposed to a teaching 
tool. Using technology as an instructional tool in the classroom may aid students in their motivation to learn 
increasingly difficult material. Today’s schools require teachers to maintain an orderly environment conducive to 
learning, we expect them to effectively help students learn a multitude of information, and now we also expect them 
to be proficient in all technological resources to engage students academically. These increasing demands will call 
for continual support from the school district in areas such as technology resources, professional development, and 
time for technology mastery. Schools that recognize exemplary teachers based on just one or two of these 
requirements may be missing important linkage for an overall exemplary program. 

The purpose of this causal comparative study was to determine if, first, teachers who are successful, as 
measured by the state assessment, are using technology in the English language arts and social studies classrooms at 
the secondary level in Region 2 public schools in South Texas. A second purpose of the study was to determine if 
there is a difference in student performance as measured by technology and the level of classroom implementation. 
The study was guided by the following questions: 1) Is there a difference in the technology implementation for 
teachers whose students score high on state assessment and those whose students score low on state assessment? and 
2) What are teachers’ levels of implementation of their instructional methods that improve student learning? 

The No Child Left Behind Act, signed into law by President George W. Bush in 2002, required states to 
increase their accountability, use research-based strategies, obtain highly qualified teachers, and narrow achievement 
gaps for all students (Kimmelman, 2006). The increasingly high standards that students must meet on state and 
federal accountability weigh heavily on the shoulders of school systems. States, such as Texas, are continually 
assessing students on higher and higher standards to meet the measures set by legislators. To successfully attain this 
mark, schools must continually assess, monitor, and use the tools which bring them the greatest success. 


Copyright by author(s); CC-BY 


64 


The Clute Institute 



Contemporary Issues In Education Research - First Quarter 2015 


Volume 8, Number 1 


This study could be significant for school leaders in helping them decipher instructional strategies that are 
meaningful and have promising results. Administrators would be able to use the study to evaluate groups of teachers 
and instructional practices they routinely use to enhance instruction and motivate students to perform well in the 
classroom and on assessments. They will be able to analyze the impact of technology in the classroom, the way in 
which the technology is implemented, and the professional development needed to support the classroom 
technology. 

METHOD 

This study was a non-experimental quantitative study. Surveys would determine trends, attitudes or 
opinions of groups. Creswell (2009) identified the survey design method procedures as one in which the researches 
“generalize or make claims about the population’’ (p.145). The survey provided quality research from the 
quantitative methods which was synthesized into a well-rounded, in-depth study (Creswell, 2009). 

As stated in the research questions, two main ideas need to be addressed. First was the quantitative data of 
the student proficiency on the state assessment scores. These data were collected through annual federal and state 
reports on student academic performance on the STAAR test. Second, the data were disaggregated to develop two 
populations of teachers - first, a group that was successful, by state standards, with their students on the STAAR test 
and proficient - defined by the researcher as a teacher scoring above 1600 in English language arts and 3700 in 
social studies. The second group included those teachers who were not successful on the STAAR test, according to 
the state standards, or scored below 1600 in English language arts and 3700 in social studies. These teachers were 
then given a quantitative survey using a Likert scale to assess the level of technology implementation that occurred 
in their classroom. 

The study was conducted with middle schools in Region 2 of South Texas. The results of this study were 
based on student test scores from the State of Texas Assessment of Academic Readiness (STAAR) and surveys from 
the teachers who taught those students during the 2011-2012 school year. The researcher used an analysis of 
variance (ANOVA) to test the null hypothesis of no significant differences in the use of technology in the classroom 
of students who are proficient on the STAAR exam and those who are non-proficient. 

Instrumentation 

Two instruments were used to conduct the research. The first was the STAAR assessment which is the state 
assessment developed by the Pearson Corporation and which is aligned to the state standards. All Texas public 
school students are required to participate in the assessment. 

The second instrument was a technology survey for teachers which was developed by Vannatta and 
O’Bannon and previously given to teachers who participated in the “Goals 2000 Pre-service Technology Infusion 
Project”. The survey evaluates the proficiency of teachers to use technology and the manner in which the technology 
is applied in the classroom. 

The Teacher Technology Integration survey consisted of four sections: 1) Risk Taking and Comfort with 
Technology, 2) Technology Access, 3) Teacher Administrative and Instructional Use, and 4) Configuration of 
Student Use. The first two sections were not used in the study. The third part of the survey measured how the 
teachers use technology in the classroom, and the last section measured the teachers’ perceptions of how students 
use technology in their classroom. This survey is based on a Likert scale of 1 -4, with 4 having the highest value. 
The options for choice were 1 =None, 2=Rarely (once or twice a month), 3=Moderately (several times per semester), 
and 4=High (Almost weekly per semester). 

A teacher’s total score for the survey was derived from multiple questions on the survey and the Likert 
scale score was added for each of the teachers. This score was divided by the proficient and non-proficient teachers. 
The totals of the proficient teachers’ scores were added together and divided by the number of proficient teachers. 
The non-proficient teachers’ scores were then added together and divided by the number of non-proficient teachers. 
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Reliability And Validity 

The survey also evaluates the capacity of the students’ use of computer and media in the classroom (Muir- 
Herzig, 2004). According to Muir-Herzig, this survey has been used numerous times with various teacher samples 
and has generated reliability coefficients (Cronback’s alpha) ranging from r=0.8185 to 1-0.9265”. 

Population And Sample 

This study focused on a number of proficient teachers (n=23) and who had students who were successful on 
the STAAR state assessment. The study additionally included a number of non-proficient teachers (n=23) whose 
students were not successful on the STAAR as measured by state standards. The teachers were from varied 
demographic backgrounds located in Region 2 in South Texas and were selected from various size school districts 
and communities. All teachers included in the survey taught at public schools. 

Procedures 

The study began with the identification of schools in Region 2 area. The researcher collected English 
language arts (ELA) and social studies scores from Region 2 Education Service Center schools from the STAAR 
assessment that was administered in the spring of 2012. Scores from the STAAR assessment are available online 
through the Texas Education Agency website. Schools were contacted to retrieve the names of teachers in the 
subject areas of ELA and social studies for survey data collection. School names and contact information were 
accessible through the Education Service Center, Region 2 website. The researcher contacted the individual teachers 
through the school mail and and systems. Teachers were mailed a survey to collect data to assess their technology 
knowledge and methods of implementing technology in the classroom and surveys were mailed back to the 
researcher within two weeks. Teacher data were confidential; the teacher’s names remained anonymous. 

Creswell (2009) stated that data analysis for a survey study consisted of compiling the reports of 
procedures gathered from the quantitative data and expounding on the development of depth of information through 
the analysis procedure. While the two types of data will be synthesized together, two individual reports will also be 
produced. 

RESULTS 

The purpose of this study was to analyze whether there was a difference in the use of technology for 
teachers who had proficient scores on STAAR tests and in the technology use of those who did not have proficient 
scores in STAAR tests. This study took place at Texas public middle schools located in Region 2 of South Texas 
and utilized student data from the State of Texas Assessment of Academic Readiness (STAAR) scores from the 
2011-2012 school year. The Teacher Technology Survey was utilized to collect from teachers who were teaching 
students linked to the STAAR scores. 

The researcher utilized two sections of the survey that related to both teacher and student use of technology 
to investigate the significant relationship between test scores and teachers who use technology in the classroom. 
Part one measured whether teachers were using technology for management purposes, such as attendance or grades. 
The second section of the survey measured how the students used technology for learning in the classroom. 
Statistics for each group were run individually and then as a combined group. In using the combined group, the 
researcher was able to add the effect size statistic for each of the dependent variables. 

Forty-six teachers participated in the technology survey. Of the 46, 23 scored proficient and 23 scored non¬ 
proficient in their STAAR scores. The mean for the proficient scores was computed by taking the average of all of 
the proficient scores and dividing by twenty-three. 

In Table 1, respondents answered questions related to teacher use of technology in the classroom. The non¬ 
proficient teachers’ mean was 52.39 with a standard deviation of 7.191, while the proficient teachers’ mean was 
47.26 with a standard deviation of 6.390. 
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The mean for the non-proficient scores was computed by taking the average of all of the non-proficient 
scores and dividing by 23. The Likert scale was used with a range of 21 to 96 on the possible scoring ability, with 36 
being the median. The results of the two independent groups are reported in Tables 1 and 2. 


Table 1: Descriptive Statistics For Teacher Use 


Test 

Mean 

Standard Deviation 

N 

Non-proficient 

52.39 

7.191 

23 

Proficient 

47.26 

6.390 

23 

Total 

49.83 

7.209 

46 


In Table 2, results were based on teachers’ answers for student use of technology in the classroom. Non¬ 
proficient teachers had a mean of 53.22 with a standard deviation of 15.765 and proficient teachers had a mean of 
40.96 with a standard deviation of 13.646. 


Table 2: Descriptive Statistics For Student Use 


Test 

Mean 

Standard Deviation 

N 

Non-proficient 

53.22 

15.765 

23 

Proficient 

40.96 

13.646 

23 

Total 

47.09 

15.842 

46 


Table 3 is the combined scores for both teachers’ and students’ technology use in the classroom. The non¬ 
proficient teachers’ mean was 105.61 with a standard deviation of 21.339 and the proficient teachers’ mean was 
88.22 with a standard deviation of 18.839. 


Test of Between-Subject Effects is reported in Table 3. 


Table 3: Descriptive Statistics For Total Technology Use From The Teacher Technology Survey 


Test 

Mean 

Standard Deviation 

N 

Non-proficient 

105.61 

21.339 

23 

Proficient 

88.22 

18.839 

23 

Total 

96.91 

21.758 

46 


After running the ANOVA for the total technology use, the Levene’s test was run to ensure that the 
assumption of homogeneity was met. Levene’s Test of Equality of Error Variances is reported in Table 4. 



Table 4: Dependent Variable: Total Technology Use-Teachers And Students 


F 

dfl 

df2 

Sig. 

.034 

1 

44 

.855 


Overall, there is a significant difference, as noted in Table 5, in the use of technology in the classroom and 
the students’ test scores [F(l,44) = 8.59, p <.01]. There was a significant difference in the STAAR scores and the 
level of implementation of technology in the classroom. Students whose teacher did not use technology were more 
successful on their STAAR exam, as measured by the Teacher Technology Survey, than those whose teacher did use 
technology. 


Table 5: Dependent Variable: Overall Technology Use-Teachers and Students 


Source 

Type III Sum 
of Squares 

df 

Mean Square 

F 

Sig. 

Partial Eta 
Squared 

Corrected Model 3475.261 a 

1 

3478.261 

8.586 

.005 

.163 

Intercept 

432038.348 

1 

432038.348 

10663439 

.000 

960 

Proficiency 

3478.261 

1 

3478.348 

8.586 

.005 

163 

Error 

17825.391 

44 

405.123 




Total 

453342.000 

46 





Corrected 

21303.652 

45 





a R Squared = 

.163 (Adjusted R Squared = 

.144) 
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The results of the proficiency test and 95% Confidence Level are shown in Table 6. 


Table 6: Proficiency And Confidence Measurement 


Proficiency 

Mean 

Standard error 

95 % confidence level 

Non-proficient 

105.609 

4.197 

Lower Bound 

97.150 

Upper Bound 

114.067 

Proficient 

88.217 

4.197 

79.759 

96.676 


RESULTS SUMMARY 

A one-way Analysis of variance was conducted to evaluate the difference between proficiency on STAAR 
testing in English language arts and social studies and the teachers’ level of implementation of technology. The 
independent variable, the teacher’s class STAAR score, included two levels: proficient and non-proficient. The 
dependent variable was the level of implementation of technology in the classroom. The ANOVA was significant, 
U(l,44) = 8.59, p = .005. The strength of the relationship between test score proficiency and teacher technology use 
in the classroom, as assessed by rf, was strong, with the test scores accounting for 16% of the variance of the 
dependent variable (Green & Salkind, 2011). 

Follow-up tests were conducted to evaluate pairwise differences among the means. Because the variance 
of the two groups ranged from 354.95 to 455.40, an assumption that the variances were homogeneous was not made 
and the Levene’s test of Equality of Error Variances was used. There was a significant difference in the means 
between the proficient and non-proficient scores. The group that received proficient scores showed a lower teacher 
use of technology. The group with non-proficient scores showed a higher teacher use of technology. The 95% 
confidence intervals for the pairwise differences, as well as the means and standard deviations for the two groups, 
are reported in Table 4 (Green & Salkind, 2011). 

On teacher use of technology, teachers of students who were not proficient on their STAAR testing had a 
higher mean score (M =52.39, SD = 7.19) than those whose students were proficient (M = 47.26, SD = 6.39). 
Additionally, the same pattern was found on the student use of technology, non-proficient students (M = 53.22, SD 
= 15.77), and for proficient students (M = 40.96, SD 13.65). The combined total for the two groups mirrored the 
individual test for the categories non-proficient (M = 105.61, SD = 21.34) and proficient (M = 96.91, SD = 18.84). 
The F-value suggested that there was a significant difference in the scores. Therefore, there was a significant 
difference in the relationship of STAAR scores and teachers who use technology in the classroom. 

DISCUSSION 

Identification of effective educational practices to enhance student learning continues to be a difficult task 
(Goe, 2007). As schools continue to implement required technology standards and look for ways to implement more 
technology, they must still be cognizant of the proficiency level of their students as they must master the state 
objectives. In this study, the use of technology by the teacher or student did not enhance their proficiency level on 
the Texas state assessment. The researcher did not survey students of the classroom which were measured. 
Additionally, teachers who completed the survey potentially have a skewed view on the level of implementation of 
their technology usage. It is difficult to determine the accuracy of the survey which was completed by the teachers. 

It appears that it is not enough to use technology by teachers or students but to understand and identify 
factors that would be able to contribute to student learning. Monitoring the use of technology for content areas or 
for technology standards may have an impact on student scores as well (Miranda & Russell, 2011). School 
administrators need to closely evaluate the technology use and progress on their campuses and evaluate the 
implementation of technology and how it relates to student test scores. The researcher identifies the need for more 
in-depth research on this subject before changing school technology programs. 
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Schools must continually find ways to meet the standards of No Child Left Behind and determine effective 
strategies to engage students in their education. The focus of many studies to improve student achievement has 
turned to technology usage and teacher effectiveness. Current research identifies many aspects of teacher 
effectiveness. Studies on instructional practices, curriculum alignment, and school culture have been identified as 
contributing to teacher effectiveness (The Gates Foundation, 2012). However, the results of this study indicated that 
there was a problem with technology usage in the classroom with regard to state assessment scores, such as STAAR. 
The results indicated that teachers who used technology in the classroom would score lower on state assessments 
than teachers who did not use technology in the classroom. 

Several variables could have affected these outcomes regarding the use of technology and assessment 
scores. Variables that could affect the final outcome of the study would be the fidelity and integrity of teachers who 
are completing the survey. The sample size of the study group could be another affected variable. Finally, the 
difference of opinions from students and teachers on the manner in which technology is used can possibly differ. 

RECOMMENDATIONS FOR PRACTITIONERS 

The findings of the study question the use of technology in instructional improvement. The researcher 
offers the following recommendations: 1) Schools should evaluate current instructional practices regarding the 
implementation of technology in the classroom, 2) Administrators could document technology methods and state 
assessment scores for correlational activities, and 3) Schools can provide strategies that will develop technology 
standards and provide purpose to the use of technology for learning. 
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